Document ranking using web evidence

Evidence based on web graph structure is reportedly used by the current generation of World-Wide Web (WWW) search engines to identify “high-quality”, “important” pages and to reject “spam” content. However, despite the apparent wide use of this evidence its application in web-based document retrieval is controversial. Confusion exists as to how to incorporate web evidence in document ranking, and whether such evidence is in fact useful. This thesis demonstrates howweb evidence can be used to improve retrieval effectiveness for navigational search tasks. Fundamental questions investigated include: which forms of web evidence are useful, how web evidence should be combined with other document evidence, and what biases are present in web evidence. Through investigating these questions, this thesis presents a number of findings regarding howweb evidence may be effectively used in a general-purpose web-based document ranking algorithm. The results of experimentation with well-known forms of web evidence on several small-to-medium collections of web data are surprising. Aggregate anchor-text measures perform well, but well-studied hyperlink recommendation algorithms are far less useful. Further gains in retrieval effectiveness are achieved for anchor-text measures by revising traditional full-text rankingmethods to favour aggregate anchor-text documents containing large volumes of anchor-text. For home page finding tasks additional gains are achieved by including a simple URL depth measure which favours short URLs over long ones. The most effective combination of evidence treats document-level and web-based evidence as separate document components, and uses a linear combination to sum scores. It is submitted that the document-level evidence contains the author’s description of document contents, and that the web-based evidence gives the wider web community view of the document. Consequently if both measures agree, and the document is scored highly in both cases, this is a strong indication that the page is what it claims to be. A linear combination of the two types of evidence is found to be particularly effective, achieving the highest retrieval effectiveness of any query-dependent evidence on navigational and Topic Distillation tasks. However, care should be taken when using hyperlink-based evidence as a direct measure of document quality. Thesis experiments show the existence of bias towards the home pages of large, popular and technology-oriented companies. Further empirical evidence is presented to demonstrate how the authorship of web documents and sites directly affects the quantity and quality of available web evidence. These factors demonstrate the need for robust methods for mining and interpreting data from the web graph.